Image down-sampling transcoding method and device

ABSTRACT

Provided are method and device for down-sampling an image from H.264 to MPEG-4 at a high speed. The image down-sampling transcoding method for converting a first macro block having a first format into a second macro block having a second format includes: (a) determining a block mode of the second macro block and converting the first macro block into the second macro block with the determined block mode, when the number of intra macro blocks included in the first macro block satisfies a first condition; (b) determining a block mode of the second macro block and converting the first macro block into the second macro block with the determined block mode, when the number for inter macro blocks included in the first macro block satisfies a second condition; and (c) re-adjusting a motion vector of the second macro block, after the conversion of the first macro block into the second macro block is finished.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image down-sampling coding methodand device, more particularly to a method and device for performing animage down-sampling process from H.264 to MPEG-4 at a high speed.

2. Description of Related Art

Thanks to the development of computers and networks, a variety of datacan be transmitted from a transmission party to a reception party. Therehave been multimedia standard compression formats developed fortransmitting large-capacity multimedia data through networks. Variousvideo transcoding methods of performing conversion processes suitablefor circumferences of the transmission party and the reception party byconsidering QoS (Quality of Service) of both parties have beenintroduced. Such video coding methods are roughly divided into atranscoding method in a pixel domain and a transcoding method in a DCT(Discrete Cosine Transform) domain.

The best transcoding method in view of image quality is a cascadedpixel-domain transcoding method of decoding input bit streams of imagesand encoding the decoded images again. However, since the encodingprocess is performed once more, such a conventional method has a problemwith the increase in complexity. Therefore, an efficient transcodingmethod which can improve the complexity while maintaining the imagequality is required.

SUMMARY OF THE INVENTION

The present invention is contrived to solve the above-mentionedproblems. An objective of the present invention is to provide an imagedown-sampling transcoding method and device which can reduce complexitywhile maintaining the image quality by re-using information on macroblocks. That is, the present invention provides a transcoding methodwhich re-uses information on macro blocks used in an H.264 decoder so asto reduce the complexity.

Another objective of the invention is to provide a transcoding methodand device for smooth communication by reducing an image bit streamhaving a CIF (Common Intermediate Format) size, which is encoded by anH.264 (ITU-T Recommendation H.264 and ISO/IEC MPEG-4 Part 10 AdvancedVideo Coding) BP (Baseline Profile) encoder, into a half in image sizein a pixel domain and converting the image bit stream into an MPEG-4 SP(Simple Profile) image bit stream having a QCIF (Quarter CommonIntermediate Format) size. Other objectives of the invention will becomeapparent through preferred embodiments to be described later.

According to a first aspect of the present invention, there is provideda method of performing an image down-sampling process from H.264 toMPEG-4 at a high speed.

According to a preferred embodiment of the invention, there is providedan image down-sampling transcoding method for converting a first macroblock having a first format into a second macro block having a secondformat, the image down-sampling transcoding method comprising the stepsof: (a) determining a block mode of the second macro block having a C×Dsize (where C and D are natural numbers) and converting the first macroblock into the second macro block with the determined block mode, whenthe number of intra macro blocks included in the first macro blockhaving a A×B size (where A and B are natural numbers) satisfies a firstcondition; (b) determining a block mode of the second macro block andconverting the first macro block into the second macro block with thedetermined block mode, when the number of inter macro blocks included inthe first macro block satisfies a second condition; and (c) re-adjustinga motion vector of the second macro block, after the conversion of thefirst macro block into the second macro block is finished.

The first macro block may be converted into the second macro block withthe determined block mode while being down-sampled using an averagefilter.

In the image down-sampling transcoding method, step (a) may includeconverting the first macro block into one intra macro block, when thefirst macro block includes only the intra macro blocks.

Here, step (a) may include determining the block mode using an SADcomparison method for macro blocks, when the first macro block includessome intra macro blocks.

In the image down-sampling transcoding method, step (b) may include:converting the first macro block into the second macro block with theinter mode when the number of inter macro blocks is greater than orequal to a predetermined number or when a difference value between themotion vectors is calculated and the difference value is less than orequal to a predetermined value; and determining the block mode using anSAD comparison method of macro blocks and converting the first blockinto the second block with the determined block mode in the other cases.

The SAD comparison method may include the steps of: calculating SADsbetween two inter macro blocks and determining the least inter SAD; andconverting the first macro block into the second macro block with theblock mode corresponding to the least SAD by comparing the least interSAD with the predetermined intra SAD.

A mean pixel value of a macro block may be calculated by the followingequation:${MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}$where org_(ij) denotes pixel values of 16×16 pixels in a macro block,and MB_(mean) denotes the mean pixel value of a macro block. Here, theintra SAD may be calculated by the following equation.${SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}$

The re-adjustment of the motion vector may be performed using thefollowing equation:${{{MV}_{i,j} = \left( \frac{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}{mv}_{m,n,k,l}}}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right)}\operatorname{>>}2},\quad i,{j = 0},1$where m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in an 8×8 mode block which is an intermediate macroblock, and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in a 4×4 mode block corresponding to the (m,n)-thmacro block in 2×2 macro blocks.

In the image down-sampling transcoding method, (c) may include the stepsof: searching for an integer pixel motion vector in ±3 neighboringpixels about an integer pixel motion vector; and searching for a halfpixel motion vector in ±1 neighboring pixels about the searched integerpixel motion vector.

According to another preferred embodiment of the invention, there isprovided an image down-sampling transcoding method for converting afirst macro block having a first format into a second macro block havinga second format, the image down-sampling transcoding method comprisingthe steps of: determining any one of the candidate block modes havingthe second format, which is determined in accordance with apredetermined condition by types of macro blocks included in the firstmacro block having a predetermined size, as a block mode of the secondmacro block and converting the first macro block into the second macroblock with the determined block mode; and re-adjusting a motion vectorof the second macro block, after the conversion of the first macro blockinto the second macro block is finished.

In the image down-sampling transcoding method, the predeterminedcondition may be such that one intra macro block having the secondformat is determined as the block mode when three or more intra macroblocks are included in the first macro block.

Furthermore, the predetermined condition may be such that any one of thecandidate block modes is determined as the block mode using a SADcomparison method of macro blocks included in the first macro block,when two or less intra macro blocks are included in the first macroblock. Here, the candidate block modes may include one or more of a skipmode, an inter 16×16 mode, and an inter 8×8 mode.

Alternatively, The predetermined condition may be such that any one ofthe candidate block modes is determined as the block mode using a SADcomparison method of macro blocks, when only inter macro blocks areincluded in the first macro block and two or less macro blocks in theinter 16×16 mode are included in the inter macro blocks. Here, thecandidate block modes may include one or more of a skip mode, an inter16×16 mode, and an inter 8×8 mode.

When a difference between the motion vectors in the respective macroblocks is calculated and the difference is less than or equal to apredetermined threshold value, any one of the inter 16×16 mode and theskip mode may be determined as the block mode and in the other cases,the inter 8×8 mode may be determined as the block mode.

Here, any one of the candidate block modes may be a block mode havingthe least SAD value.

A mean pixel value of a macro block may be calculated by the followingequation:${MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}$where org_(ij) denotes pixel values of 16×16 pixels in a macro block andMB_(mean) denotes the mean pixel value of a macro block. The intra SADmay be calculated by the following equation:${SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}$

The re-adjustment of the motion vector may be performed using thefollowing equation:${{{MV}_{i,j} = \left( \frac{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}{mv}_{m,n,k,l}}}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right)}\operatorname{>>}2},\quad i,{j = 0},1$where m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in an 8×8 mode block which is an intermediate macroblock and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in a 4×4 mode block corresponding to the (m,n)-thmacro block in 2×2 macro blocks.

According to another aspect of the invention, there is provided a devicefor down-sampling an image from H.264 to MPEG-4 at a high speed.

That is, according to a preferred embodiment of the invention, there isprovided an image down-sampling transcoding device for converting afirst macro block having a first format into a second macro block havinga second format, the image down-sampling transcoding device comprising:a first conversion unit determining a block mode of the second macroblock and converting the first macro block into the second macro blockwith the determined block mode, when the number of intra macro blocksincluded in the first macro block satisfies a first condition; a secondconversion unit determining a block mode of the second macro block andconverting the first macro block into the second macro block with thedetermined block mode, when the number for inter macro blocks includedin the first macro block satisfies a second condition; and a motionvector re-adjusting unit re-adjusting a motion vector of the secondmacro block, after the conversion of the first macro block into thesecond macro block is finished.

The first conversion unit may convert the first macro block into oneintra macro block, when the first macro block includes only the intramacro blocks and may determine the block mode using an SAD comparisonmethod for macro blocks, when the first macro block includes some intramacro blocks.

The second conversion unit may convert the first macro block into thesecond macro block with the inter mode, when the number of inter macroblocks is greater than or equal to a predetermined number or when adifference value between the motion vectors is calculated and thedifference value is less than or equal to a predetermined value and maydetermine the block mode using an SAD comparison method of macro blocksand converts the first block into the second block with the determinedblock mode in the other cases.

The SAD comparison method is characterized by: calculating SADs betweentwo inter macro blocks and determining the least inter SAD; andconverting the first macro block into the second macro block with theblock mode corresponding to the least SAD by comparing the least interSAD with the predetermined intra SAD.

According to another preferred embodiment of the invention, there isprovided an image down-sampling transcoding device for converting afirst macro block having a first format into a second macro block havinga second format, the image down-sampling transcoding device comprising:a conversion unit determining any one of candidate block modes havingthe second format, which is determined in accordance with apredetermined condition by types of macro blocks included in the firstmacro block having a predetermined size, as a block mode of the secondmacro block and converting the first macro block into the second macroblock with the determined block mode; and a motion vector re-adjustingunit re-adjusting a motion vector of the second macro block, after theconversion of the first macro block into the second macro block isfinished.

The predetermined condition may be such that one intra macro blockhaving the second format is determined as the block mode when three ormore intra macro blocks are included in the first macro block.

The predetermined condition may be such that any one of the candidateblock modes is determined as the block mode using a SAD comparisonmethod of macro blocks included in the first macro block, when two orless intra macro blocks are included in the first macro block. Here, thecandidate block modes may include one or more of a skip mode, an inter16×16 mode, and an inter 8×8 mode.

Alternatively, the predetermined condition may be such that any one ofthe candidate block modes is determined as the block mode using a SADcomparison method of macro blocks, when only inter macro blocks areincluded in the first macro block and two or less macro blocks in theinter 16×16 mode are included in the inter macro blocks. Here, thecandidate block modes may include one or more of a skip mode, an inter16×16 mode, and an inter 8×8 mode.

The conversion unit may calculate a difference between the motionvectors in the respective macro blocks, may determine any one of theinter 16×16 mode and the skip mode as the block mode when the differenceis less than or equal to a predetermined threshold value, and in theother cases, may determine the inter 8×8 mode as the block mode.

A mean pixel value of a macro block may be calculated by the followingequation:${MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}$where org_(ij) denotes pixel values of 16×16 pixels in a macro block andMB_(mean) denotes the mean pixel value of a macro block. Here, the intraSAD may be calculated by the following equation:${SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}$

The re-adjustment of the motion vector may be performed using thefollowing equation:${{{MV}_{i,j} = \left( \frac{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}{mv}_{m,n,k,l}}}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right)}\operatorname{>>}2},\quad i,{j = 0},1$where m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in an 8×8 mode block which is an intermediate macroblock, and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in a 4×4 mode block corresponding to the (m,n)-thmacro block in 2×2 macro blocks.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become more apparent by describing in detail preferred embodimentsthereof with reference to the attached drawings in which:

FIG. 1 is a diagram illustrating an image down-sampling method accordingto a preferred embodiment of the invention;

FIGS. 2A to 2D are diagrams illustrating a conversion method of macroblock modes and motion vectors according to a preferred embodiment ofthe invention;

FIG. 3 is a block diagram illustrating a configuration of a transcodingdevice for down-sampling an image according to a preferred embodiment ofthe invention;

FIG. 4 is a flow diagram illustrating an image down-sampling methodaccording to a preferred embodiment of the invention;

FIGS. 5A to 5D are graphs illustrating an image down-sampling methodaccording to a preferred embodiment of the invention; and

FIGS. 6A to 6D are diagrams illustrating a conversion method of macroblock modes and motion vectors according to another preferred embodimentof the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, image down-sampling transcoding method and device accordingto exemplary embodiments of the present invention will be described indetail with reference to the attached drawings such that the inventioncan be easily put into practice by those skilled in the art In thedrawings, like elements are denoted by like reference numerals and thusrepeated description thereof is omitted.

Generally, the simplest method for converting macro blocks (MB) in anH.264 frame into macro blocks in an MPEG-4 frame is a cascadedpixel-domain transcoding method of decompressing the compressed H.264frame to be input and then compressing again the decompressed frame inaccordance with MPEG-4. however, in such a method, since a motionestimation process of all the macro blocks in the whole frame isperformed by an MPEG-4 encoder, real-time transmission by a transcodercan be hindered due to a large amount of calculation.

In the invention, block modes and motion vector data used in the H.264decoder are re-used by the MPEG-4 encoder so as to perform the real-timetransmission without performing the motion estimation process. Accordingto an embodiment of the invention, a format, macro block data, andmotion vector data of a current frame are extracted in the course ofperforming a variable length decoding (VLD) process of a bit streamcompressed in accordance with H.264. Here, when the decompressed bitstream has a P (Predictive) frame format, the motion estimation processfor which the MPEG-4 encoder requires much time can be omitted byreusing the block types and the motion vectors of the macro blocks.

As described above, a transcoding device according to the invention caninclude the H.264 decoder and the MPEG-4 encoder and the data extractedby the H.264 decoder can be supplied to the MPEG-4 encoder. Thetranscoding device according to the invention can be provided in a mediagateway server and the like for transmitting bit streams to a receptionunit.

Hereinafter, a transcoding method performed by the transcoding deviceaccording to the invention will be mainly described. Configurations andoperations of the transcoding device according to the invention can beeasily understood by those skilled in the art through the followingdescription.

Now, a transcoding method employing conversion between block types in apixel domain and adjustment of motion vectors at the time of conversionof H.264 BP (Baseline Profile) into MPEG-4 SP (Simple Profile) will bedescribed with reference to FIG. 1 and FIGS. 2A to 2E. Then, theconfiguration of the transcoding device will be described with referenceto FIG. 3 and a specific transcoding method will be then described withreference to FIG. 4. Results of a test of transcoding the H.264 BP intothe MPEG-4 SP by the use of a pixel-domain transcoding method will bedescribed with reference to FIGS. 5A to 5D. Finally, a method of rapidlyperforming a transcoding process using the conversion between blocktypes and the adjustment of motion vectors through the use of astatistical technique will be described with reference to FIGS. 6A to6D.

Procedure of Image Down-Sampling Transcoding

An image down-sampling method using an average filter as an imagedown-sampling method according to an exemplary embodiment of theinvention is shown in FIG. 1.

For example, when a four-MB (Macro Block) type is converted into aone-MB type, a source image conventionally undergoes two imageconversion processes in the course of converting an image having a CIF(Common Intermediate Format) size, which has passed through the H.264(SP) encoder, into an image having a QCIF (Quarter Common IntermediateFormat) size, thereby causing deterioration in image quality.

However, according to an embodiment of the invention, four 16×16 macroblocks in a CIF image decompressed by the H.264 decoder are down-sampledinto one 16×6 macro block, which is a QCIF image, to be encoded by theMPEG-4 encoder. According to the invention, among a method of samplingone pixel value among four pixel values by the use of the average filterand a sub-sampling method of selecting only one pixel value among fourpixel values, the method of down-sampling an image by the use of theaverage filter is used to down-sample the CIF image into the QCIF image.When an image is down-sampled using the sub-sampling method, it isslightly advantageous in view of temporal complexity, but deteriorationin image quality occurs remarkably. Accordingly, the method ofdown-sampling by the use of the average filter is used. The inventionuses the transcoding method in the pixel domain, instead of thetranscoding method in the DCT domain having low complexity ofcalculation. Since an H.264 non-linear loop filter is used in thetranscoding method in the DCT domain, a loop-filtered image should besubjected again to the DCT process and then the transcoding process inthe DCT domain and a conversion relation between a 4×4 integer DCT andan 8×8 DCT should be derived. Accordingly, the amount of calculationincreases.

H.264 uses ¼ pixel motion estimation motion and motion compensation,seven variable blocks (for example, inter 16×16, inter 16×8, inter 8×16,inter 8×8, inter 8×4, inter 4×8, and inter 4×4), and ten blockmodes.(for example, skip, intra 16×16, intra 4×4, inter 16×16, inter16×8, inter 8×16, inter 8×8, inter 8×4, inter 4×8, and inter 4×4).However, since MPEG-4 supports an intra 16×16 mode, an inter 16×16 mode,an inter 8×8 mode, and a skip mode, conversion between block types isperformed to reuse such data in MPEG-4 in the invention.

A method of converting four macro block modes and motion vectors inH.264 into a macro block mode and motion vectors in MPEG-4 according toan exemplary embodiment of the invention is shown in FIGS. 2A to 2E.

Now, a transcoding method using the conversion between block types andthe adjustment of motion vectors in the pixel domain at the time ofconversion of H.264 BP into MPEG-4 SP will be described with referenceto a variety of embodiments of converting 2×2 macro blocks of thedecompressed H.264 bit stream into one macro block which can be reusedin MPEG-4.

In FIG. 2A, the skip mode in H.264 indicates a pre-processing step forconverting 2×2 macro blocks in H.264 into one macro block in MEPG-4 andserves to convert the macro blocks with the inter 16×16 mode in H.264.

As in FIG. 2B, when the 2×2 macro blocks in H.264 includes only macroblocks having the intra 16×16 mode or the intra 4×4 mode, the 2×2 macroblocks is converted into one macro block having the intra 16×16 mode andan image is down-sampled by the use of the average filter.

As in FIG. 2C, when two or three macro blocks among the 2×2 macro blocksin H.264 are intra mode macro blocks, it is not clear which mode amongthe inter 16×16 mode, the inter 8×8 mode, and the intra 16×16 mode theMPEG-4 encoder should use.

In this case, as in the related art, when the conversion is performedwith any specific mode (the inter 16×16 mode, the inter 8×8 mode, or theintra 16×16 mode) in MPEG-4, the image quality deteriorates and theamount of encoded bits increases.

Accordingly, in the invention, a block mode having the least SAD (Sum ofAbsolute Difference) value is selected by comparing the SAD value of themacro block with the inter 16×16 mode with the sun of the SAD values offour macro blocks in the inter 8×8 mode in the MPEG-4 encoder.Subsequently, by comparing the SAD value of the selected inter mode withthe SAD value of the intra mode calculated by Expression 1 andExpression 2 described below, the block mode having the least SAD valueis finally selected. $\begin{matrix}{{MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}} & {{Expression}\quad 1} \\{{SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}} & {{Expression}\quad 2}\end{matrix}$Here, org_(ij) denotes pixel values of 16×16 pixels in a macro block andMB_(mean) denotes a mean value of the pixel values in the macro block.By performing the conversion with the optimal mode, the requirement forimage quality and the requirement for the amount of encoded bits canboth be satisfied.

As in FIG. 2D, when the 2×2 macro blocks in H.264 include four or threemacro blocks in the inter 16×16 mode, the MPEG-4 encoder performs theconversion with the inter 16×16 mode. Conventionally, when four or threemacro blocks in the inter 16×16 mode are included in the 2×2 macroblocks in H.264, the MPEG-4 encoder can perform the conversion with theinter 16×16 mode or the inter 8×8. However, in the invention, theconversion is performed with the inter 16×16 mode, not the inter 8×8mode. By first acquiring the averages of ¼ pixel motion vectors of therespective 2×2 macro blocks as integer pixel motion vectors of the fourmacro blocks in the 8×8 mode in MPEG-4 and then acquiring the average ofthe acquired integer pixel motion vectors as a motion vector of onemacro block in the inter 16×16 mode, the mode conversion is performed. Aspecific equation for acquiring the integer pixel motion vectors isexpressed as Expression 3 described below. $\begin{matrix}{{{MV}_{i,j} = {\left( \frac{{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}m}},n,k,l}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right) ⪢ 2}},\quad i,{j = 0},1} & {{Expression}\quad 3}\end{matrix}$Here, m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in the 8×8 mode block which is an intermediate macroblock and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in the 8×8 mode block which is the (m,n)-th macroblock in the 2×2 macro blocks.

The reason for performing the conversion with the inter 16×16 modeinstead of the inter 8×8 mode is as follows. It is supposed that aquantization parameter is set to 10 with respect to a foreman image atthe time of transcoding from H.264 to MPEG-4 and all the conditions areset to the same for all the conversions. Then, when the conversion isperformed with the inter 8×8 mode under the condition shown in FIG. 2D,the PSNR is enhanced by 0.08 dB from 31.61 dB to 31.69 dB in comparisonwith the conversion with the inter 16×16 mode. However, since the amountof encoded bits increases from 143.2 kps to 156.8 kps, the wholeencoding efficiency decreases.

Referring to FIG. 2E, when the 2×2 macro blocks has complex block modes,that is, one or more macro blocks in the inter 8×8 mode, one or lessmacro block in the intra mode, and macro blocks in other inter modes,the respective macro blocks of the 2×2 macro blocks are converted intothe macro blocks in the inter 8×8 mode in MPEG-4. The integer pixelmotion vectors of the macro blocks in the inter 8×8 mode in MPEG-4 areacquired by calculating the average of the ¼ pixel motion vectors ofsixteen blocks in the 4×4 mode and dividing the average by 4. Thecalculating equation is equal to Expression 3.

FIG. 3 is a block diagram illustrating a configuration of a transcodingdevice according to an exemplary embodiment of the invention.

Referring to FIG. 3, the transcoding device 300 according to theinvention includes a first conversion unit 310, a second conversion unit320, and a motion vector adjusting unit 330.

When the number of intra macro blocks included in a first macro blocksatisfies a first condition, the first conversion unit 310 determines ablock mode of a second macro block and converts the first macro blockinto the second macro block with the determined block mode. For example,the first conversion unit 310 converts the intra macro blocks into oneintra macro block (see FIG. 2B) when the first macro block includes onlythe intra macro blocks and determines the block mode by the use of theSAD comparison method of a macro block (see FIG. 2C) when the firstmacro block includes some intra macro blocks.

When the number of inter macro blocks included in the first macro blocksatisfies a second condition, the second conversion unit 320 determinesthe block mode of the second macro block and converts the first macroblock into the second macro block with the determined block mode. Forexample, the second conversion unit 320 converts the first macro blockinto the second macro block with the inter mode when the number of intermacro blocks is greater than or equal to a predetermined number or whena difference value between the motion vectors is calculated and thedifference value is less than or equal to a predetermined value, anddetermines the block mode by the use of the SAD comparison method ofmacro blocks and converts the first block into the second block with thedetermined block mode in the other cases. More specific functions willbe described in detail with reference to FIG. 4.

The motion vector adjusting unit 330 re-adjusts the motion vectors ofthe second macro block.

FIG. 4 is a flow diagram illustrating a procedure of performing thetranscoding of 2×2 macro blocks from H.264 to MPEG-4.

As shown in FIG. 4, when the 2×2 macro blocks in H.264 are input (S410),the 2×2 MB type in H.264 is converted into a 1×1 MB type in accordancewith the conditions (S420) and then the down-sampling is performed bythe use of the average filter (S430).

Thereafter, a block mode in MPEG-4 is determined.

In S425, when the number of intra macro blocks included in the 2×2 macroblocks in H.264 is greater than 2, it is determined in S430 whether thenumber of macro blocks is 4. When the number of macro blocks is not 4,the conversion with an intra mode is performed in S435 and in the othercase, S455 is performed.

On the other hand, when it is determined in S425 that the number ofintra macro blocks included in the 2×2 macro blocks in H.264 is lessthan or equal to 2, S440 is performed. When it is determined in S440that the number of macro blocks in the inter 16×16 mode included in the2×2 macro blocks in H.264 is greater than 2 (for example, when thenumber is 4 or 3), the conversion with the inter 16×6 mode is performedin S445. On the contrary, when it is determined in S440 that the numberof macro blocks in the inter 16×16 included in the 2×2 macro blocks inH.264 is less than or equal to 2, the difference values between all theinteger pixel motion vectors are compared with a predetermined thresholdvalue (for example, 4) in S450.

When the difference values are greater than or equal to the thresholdvalue, the MB mode is determined by the use of the SAD process in S455.However, when the difference values are less than the threshold value,the conversion with the inter 16×16 mode is performed in S445.

Here, the reason for performing the conversion with the inter 16×16 modeinstead of the inter 8×8 mode as the block mode to be used in the MPEG-4encoder, like the inter 8×8 mode, the inter 8×4 mode, and the inter 4×4mode in the H.264, is that the encoding and the image down-sampling inH.264 are a kind of low pass filtering (LPF).

Thereafter, the determination of the macro block mode is finished inS460 and then the conversion of a macro block is started in S415.

In this way, when the determination of the block mode is finished, there-adjustment of a motion vector is performed in S470. That is, bysearching for an integer pixel motion vector in ±3 neighboring pixelsabout a calculated integer pixel motion vector and searching for a halfpixel motion vector in ±1 neighboring pixels about the searched integerpixel motion vector, the re-adjustment of a motion vector is performed,thereby obtaining enhancement in PSNR (Peak Signal-to-Noise Ratio).

FIGS. 5A to 5D are graphs illustrating results of a test of transcodingthe H.264 BP to the MPEG-4 SP by the using the pixel-domain transcodingmethod according to an exemplary embodiment of the invention.

The test was performed using a transcoder including a JM (Joint Model)decoder reconstructed with the H.264 BP and a MoMuSys-FDIS-V1.0 encoderreconstructed with the MPEG-4 SP. The test was performed with a PentiumIV 2.8 GHz PC. H.264 bit streams generated by compressing Foreman, News,Paris, and Coast CIF (352×288) images was used in the JM encoder. In therespective images used for generation of the bit streams, an I (Intra)frame was used for only the first frame, a P (Predictive) frame was usedfor all the other frames, and 300 sequences were compressed for the testwith a frame rate of 30 Hz.

FIGS. 5A to 5B show PSNR differences in brightness values Y between thecascaded pixel-domain transcoding method according to the related art(510, 520, 530, and 540) and the image down-sampling transcoding methodaccording to the invention (515, 525, 535, and 545) at the time ofconversion of Coast, Paris, News, and Foreman image sequences from H.264to MPEG-4. Here, all the motion images other than the Foreman image ofFIG. 5D having the scene change do not exhibit deterioration in imagequality and the enhancement in image quality by 0.5 dB in maximum isobtained in the Coast image of FIG. 5A.

Table 1 shows temporal gains in the transcoding method according to theinvention. The speed-up effect of about 1,72 times is obtainedaveragely, slightly different depending upon the images. Consideringonly the MPEG-4 encoder, the speed-up effect of about 4.1 times isobtained. TABLE 1 Cascade Pixel-domain Transcoding Proposed TranscodingTrans- Trans- H.264 MPEG-4 coder H.264 MPEG-4 coder decoder encoderTotal decoder encoder Total Sequence time time time time time time News34.14 52.23 86.37 34.14 12.1 46.24 Paris 37.2 52.18 89.38 37.2 12.3 49.5Coast 51.48 53.54 105.02 51.48 13.29 64.77 Foreman 49.54 53.49 103.0349.54 13.32 62.86

As described above, the information conversion method for reusing blockinformation defined in H.264 is provided. By reusing the previous motionvectors and the block information in H.264, enhancement in performanceof about 1.8 times is obtained and the deterioration in image qualitylittle occurs, in comparison with the conventional cascaded pixel-domaintranscoding method requiring much time.

Hitherto, the method of converting the H.264 BP into the MPEG-4 SPaccording to the invention has been described. In converting the blockmode defined in H.264 into the block mode defined in MPEG-4, all theblock modes defined in MPEG-4 are considered.

However, from the point of statistical view, only the least MPEG-4 blockmodes can be used by considering the features of the 2×2 macro blocks inH.264 in converting the 2×2 macro blocks in H.264 into a 1×1 macroblock. Accordingly, it is possible to rapidly perform the transcodingwithout the deterioration of image quality.

Table 2 statistically shows conversion frequencies to the MPEG-4 blockmodes in consideration of the features of the 2×2 macro blocks in H.264.FIGS. 6A to 6D show a conversion process of macro blocks and motionvectors in consideration of the statistical method. TABLE 2 MB modes ofMPEG-4 2 × 2 MB modes of INTER INTER H.264 INTRA SKIP 16 × 16 8 × 8Total Others two Inter 16 × 16, two blocks 0.20% 13.53% 3.48% 1.88%19.09% See FIG. 6A smaller than Inter 16 × 8 or Inter 8 × 16 one Inter16 × 16, three blocks 0.22% 13.91% 2.16% 3.21% 19.49% smaller than Inter16 × 8 or Inter 8 × 16 four blocks smaller 0.14% 8.55% 0.86% 3.25%12.79% than Inter 16 × 8 or Inter 8 × 16 four Inter 16 × 16 0.26% 3.48%25.94% 0.09% 29.77% See FIG. 6B three Inter 16 × 16, one block 0.13%8.01% 5.75% 0.47% 14.35% smaller than Inter16 × 8 or Inter 8 × 16 oneIntra, three Inter 0.03% 1.15% 0.32% 1.26% 2.76% See FIG. 6C two Intra,two Inter 0.02% 0.37% 0.10% 0.65% 1.14% three Intra, one Inter 0.00%0.10% 0.04% 0.16% 0.31% See FIG. 6D four Intra 0.05% 0.09% 0.01% 0.14%0.29% Total Number 1.04% 49.19% 38.65% 11.11% 100.00%

As can be seen from Table 2 and FIG. 6, in the transcoding methodaccording to the invention, only the MPEG-4 block modes having arelatively high frequency at the time of converting the macro blockshaving a predetermined size from H,264 to MPEG-4 can be considered. Atthis time, the features of the 2×2 macro blocks in H.264 can be used.

For example, when the 2×2 macro blocks in H.264 include two macro blocksin the inter 16×16 mode and the other two macro blocks have a smallermode than the inter 16×8 mode or the inter 8×16 mode, only the inter16×16 mode, the skip mode, or the inter 8×8 mode in MPEG-4 can beconsidered for the conversion as shown in FIG. 6A. It can be seen fromTable 2 that the corresponding block modes have relatively highfrequencies. As described above, it can be determined as follows whetherone of the inter 16×16 mode and the skip mode in MPEG-4 should beselected or the inter 8×8 mode should be selected. That is, it can beprimarily determined on the basis whether the difference values betweenthe ¼ pixel motion vectors is less than a threshold value (for example,9).

When the 2×2 macro blocks in H.264 include three or more intra macroblocks, the MPEG-4 block mode having the highest conversion frequency inTable 2 is the inter 8×8 block mode. However, since the very low valuein frequency (that is, 0.14% or 0.16%) can be neglected, the intra blockmode can be determined as the MPEG-4 block mode as shown in FIG. 6D.

According to the invention described above, it is possible to providethe image down-sampling transcoding method and device which can reducethe complexity while maintaining the image quality by re-using theinformation on the macro blocks.

In addition, the transcoder for converting a macro block from H.264 toMPEG-4 can permit the use of images compressed in accordance with H.264,which is a new standard for compressing a moving picture, withoutreplacing devices employing an existing MPEG-4 codec

Although the exemplary embodiments of the present invention have beendescribed, the present invention is not limited to the embodiments, butit can be understood by those skilled in the art that the invention maybe modified in various forms without departing from the spirit and scopeof the appended claims.

1. An image down-sampling transcoding method for converting a firstmacro block having a first format into a second macro block having asecond format, the image down-sampling transcoding method comprising:(a) determining a block mode of the second macro block having a C×D size(where C and D are natural numbers) and converting the first macro blockinto the second macro block with the determined block mode, when thenumber of intra macro blocks included in the first macro block having aA×B size (where A and B are natural numbers) satisfies a firstcondition; (b) determining a block mode of the second macro block andconverting the first macro block into the second macro block with thedetermined block mode, when the number for inter macro blocks includedin the first macro block satisfies a second condition; and (c)re-adjusting a motion vector of the second macro block, after theconversion of the first macro block into the second macro block isfinished.
 2. The image down-sampling transcoding method according toclaim 1, wherein the first macro block is converted into the secondmacro block with the determined block mode while being down-sampled bythe use of an average filter.
 3. The image down-sampling transcodingmethod according to claim 1, wherein (a) includes converting the firstmacro block into one intra macro block, when the first macro blockincludes only the intra macro blocks.
 4. The image down-samplingtranscoding method according to claim 1, wherein (a) includesdetermining the block mode by the use of an SAD comparison method formacro blocks, when the first macro block includes some intra macroblocks.
 5. The image down-sampling transcoding method according to claim1, wherein (b) includes: converting the first macro block into thesecond macro block with the inter mode when the number of inter macroblocks is greater than or equal to a predetermined number or when adifference value between the motion vectors is calculated and thedifference value is less than or equal to a predetermined value; anddetermining the block mode by the use of an SAD comparison method ofmacro blocks and converting the first block into the second block withthe determined block mode in the other cases.
 6. The image down-samplingtranscoding method according to claim 4 or 5, wherein the SAD comparisonmethod includes: calculating SADs between two inter macro blocks anddetermining the least inter SAD; and converting the first macro blockinto the second macro block with the block mode corresponding to theleast SAD by comparing the least inter SAD with the predetermined intraSAD.
 7. The image down-sampling transcoding method according to claim 6,wherein a mean pixel value of a macro block is calculated by thefollowing expression:${MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}$where org_(ij) denotes pixel values of 16×16 pixels in a macro block andMB_(mean) denotes the mean pixel value of a macro block, and wherein theintra SAD is calculated by the following expression.${SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}$8. The image down-sampling transcoding method according to claim 1,wherein the re-adjustment of the motion vector is performed by the useof the following expression:${{MV}_{i,j} = {\left( \frac{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}{mv}_{m,n,k,l}}}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right) ⪢ 2}},\quad i,{j = 0},1$where m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in an 8×8 mode block which is an intermediate macroblock and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in a 4×4 mode block corresponding to the (m,n)-thmacro block in 2×2 macro blocks.
 9. The image down-sampling transcodingmethod according to claim 1, wherein (c) includes: searching for aninteger pixel motion vector in ±3 neighboring pixels about an integerpixel motion vector; and searching for a half pixel motion vector in ±1neighboring pixels about the searched integer pixel motion vector. 10.An image down-sampling transcoding method for converting a first macroblock having a first format into a second macro block having a secondformat, the image down-sampling transcoding method comprising:determining any one of candidate block modes having the second format,which is determined in accordance with a predetermined condition bytypes of macro blocks included in the first macro block having apredetermined size, as a block mode of the second macro block andconverting the first macro block into the second macro block with thedetermined block mode; and re-adjusting a motion vector of the secondmacro block, after the conversion of the first macro block into thesecond macro block is finished.
 11. The image down-sampling transcodingmethod according to claim 10, wherein the predetermined condition isthat one intra macro block having the second format is determined as theblock mode when three or more intra macro blocks are included in thefirst macro block.
 12. The image down-sampling transcoding methodaccording to claim 10, wherein the predetermined condition is that anyone of the candidate block modes is determined as the block mode by theuse of a SAD comparison method of macro blocks included in the firstmacro block, when two or less intra macro blocks are included in thefirst macro block, and wherein the candidate block modes include one ormore of a skip mode, an inter 16×16 mode, and an inter 8×8 mode.
 13. Theimage down-sampling transcoding method according to claim 10, whereinthe predetermined condition is that any one of the candidate block modesis determined as the block mode by the use of a SAD comparison method ofmacro blocks, when only inter macro blocks are included in the firstmacro block and two or less macro blocks in the inter 16×16 mode areincluded in the inter macro blocks, and wherein the candidate blockmodes include one or more of a skip mode, an inter 16×16 mode, and aninter 8×8 mode.
 14. The image down-sampling transcoding method accordingto claim 12 or 13, wherein when a difference between the motion vectorsin the respective macro blocks is calculated and the difference is lessthan or equal to a predetermined threshold value, any one of the inter16×16 mode and the skip mode is determined as the block mode and in theother cases, the inter 8×8 mode is determined as the block mode.
 15. Theimage down-sampling transcoding method according to claim 12 or 13,wherein any one of the candidate block modes is a block mode having theleast SAD value.
 16. The image down-sampling transcoding methodaccording to claim 12, wherein a mean pixel value of a macro block iscalculated by the following expression:${MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}$where org_(ij) denotes pixel values of 16×16 pixels in a macro block andMB_(mean) denotes the mean pixel value of a macro block, and wherein theintra SAD is calculated by the following expression.${SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}$17. The image down-sampling transcoding method according to claim 10,wherein the re-adjustment of the motion vector is performed by the useof the following expression:${{MV}_{i,j} = {\left( \frac{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}{mv}_{m,n,k,l}}}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right) ⪢ 2}},\quad i,{j = 0},1$where m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in an 8×8 mode block which is an intermediate macroblock and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in a 4×4 mode block corresponding to the (m,n)-thmacro block in 2×2 macro blocks.
 18. An image down-sampling transcodingdevice for converting a first macro block having a first format into asecond macro block having a second format, the image down-samplingtranscoding device comprising: a first conversion unit determining ablock mode of the second macro block and converting the first macroblock into the second macro block with the determined block mode, whenthe number of intra macro blocks included in the first macro blocksatisfies a first condition; a second conversion unit determining ablock mode of the second macro block and converting the first macroblock into the second macro block with the determined block mode, whenthe number for inter macro blocks included in the first macro blocksatisfies a second condition; and a motion vector re-adjusting unitre-adjusting a motion vector of the second macro block, after theconversion of the first macro block into the second macro block isfinished.
 19. The image down-sampling transcoding device according toclaim 18, wherein the first conversion unit: converts the first macroblock into one intra macro block, when the first macro block includesonly the intra macro blocks; and determines the block mode by the use ofan SAD comparison method for macro blocks, when the first macro blockincludes some intra macro blocks.
 20. The image down-samplingtranscoding device according to claim 18, wherein the second conversionunit: converts the first macro block into the second macro block withthe inter mode, when the number of inter macro blocks is greater than orequal to a predetermined number or when a difference value between themotion vectors is calculated and the difference value is less than orequal to a predetermined value; and determines the block mode by the useof an SAD comparison method of macro blocks and converts the first blockinto the second block with the determined block mode in the other cases.21. The image down-sampling transcoding device according to claim 20,wherein the SAD comparison method includes: calculating SADs between twointer macro blocks and determining the least inter SAD; and convertingthe first macro block into the second macro block with the block modecorresponding to the least SAD by comparing the least inter SAD with thepredetermined intra SAD.
 22. An image down-sampling transcoding devicefor converting a first macro block having a first format into a secondmacro block having a second format, the image down-sampling transcodingdevice comprising: a conversion unit determining any one of candidateblock modes having the second format, which is determined in accordancewith a predetermined condition by types of macro blocks included in thefirst macro block having a predetermined size, as a block mode of thesecond macro block and converting the first macro block into the secondmacro block with the determined block mode; and a motion vectorre-adjusting unit re-adjusting a motion vector of the second macroblock, after the conversion of the first macro block into the secondmacro block is finished.
 23. The image down-sampling transcoding deviceaccording to claim 22, wherein the predetermined condition is that oneintra macro block having the second format is determined as the blockmode when three or more intra macro blocks are included in the firstmacro block.
 24. The image down-sampling transcoding device according toclaim 22, wherein the predetermined condition is that any one of thecandidate block modes is determined as the block mode by the use of aSAD comparison method of macro blocks included in the first macro block,when two or less intra macro blocks are included in the first macroblock, and wherein the candidate block modes include one or more of askip mode, an inter 16×16 mode, and an inter 8×8 mode.
 25. The imagedown-sampling transcoding device according to claim 22, wherein thepredetermined condition is that any one of the candidate block modes isdetermined as the block mode by the use of a SAD comparison method ofmacro blocks, when only inter macro blocks are included in the firstmacro block and two or less macro blocks in the inter 16×16 mode areincluded in the inter macro blocks, and wherein the candidate blockmodes include one or more of a skip mode, an inter 16×16 mode, and aninter 8×8 mode.
 26. The image down-sampling transcoding device accordingto claim 24 or 25, wherein the conversion unit calculates a differencebetween the motion vectors in the respective macro blocks, determinesany one of the inter 16×16 mode and the skip mode as the block mode whenthe difference is less than or equal to a predetermined threshold value,and in the other cases, determines the inter 8×8 mode as the block mode.27. The image down-sampling transcoding device according to claim 24,wherein a mean pixel value of a macro block is calculated by thefollowing expression:${MB}_{mean} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{org}_{i,j}}}$where org_(ij) denotes pixel values of 16×16 pixels in a macro block andMB_(mean) denotes the mean pixel value of a macro block, and wherein theintra SAD is calculated by the following expression.${SAD}_{INTRA} = {\sum\limits_{i = 0}^{15}{\sum\limits_{j = 0}^{15}{{{org}_{i,j} - {MB}_{mean}}}}}$28. The image down-sampling transcoding device according to claim 22,wherein the re-adjustment of the motion vector is performed by the useof the following expression:${{MV}_{i,j} = {\left( \frac{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}{mv}_{m,n,k,l}}}{\sum\limits_{k = 0}^{3}{\sum\limits_{l = 0}^{3}l}} \right) ⪢ 2}},\quad i,{j = 0},1$where m and n denote coordinate values in the axis of abscissa and theaxis of ordinate in an 8×8 mode block which is an intermediate macroblock and k and l denote coordinate values in the axis of abscissa andthe axis of ordinate in a 4×4 mode block corresponding to the (m,n)-thmacro block in 2×2 macro blocks.